The present Invention relates to a technology for demultiplexing a multiplexed data in which an image data and a control data are multiplexed, into an image data and a control data, and controlling an apparatus in response to the demultiplexed image data while being synchronized with reproduction of the demultiplexed image data.
Conventionally, as a technology for demultiplexing an image data and a control data from each other, as disclosed in JP-A-22719/1993 for example, a technology is known, in which image, voice and control data are multiplexed, and the data are transferred, multiplexedly demultiplexed, and utilized.
FIG. 7 is a block diagram for explaining such a prior art. In this figure, a multimedia multiplexing and demultiplexing/inter-terminal controlling section 71 conducts multiplexing and demultiplexing of an image data and a control data in a communication frame which was transmitted and received by a network interface communication controlling section 72.
An image codec 73 conducts encoding and decoding of an image data. A video inputting and outputting interface 74 connects an external camera 75 with the image codec 73. A control section 76 controls operation of an external camera 78 through a camera external instrument multiconnector 77 based on a control data received from the multimedia multiplexing and demultlplexing/inter-terminal controlling section 71.
According to such an arrangement, if the camera 75 is connected with a monitor through the video inputting and outputting interface 74, it becomes to be possible to structure a television conference system capable of controlling operation of the camera from an outside.
By the way, in such a prior art, timing of image reproduction cannot be synchronized with processing timing of a control data. Accordingly, there were a task in which an image display by means of an image data, a voice data and a control data and a control device operate independently, and a series of these operations cannot be synchronized with each other, and a task in which, since respective processing sections are independent of each other, when image reproduction and device control are actually conducted, a time lag occurs due to a difference between processing periods of time, and it is not possible to synchronously control the operation of the camera In accordance with a certain image.
Therefore, the objective of the present invention is to solve the above-described tasks.
Also, the objective of the present invention is to provide a technology of multiplex and demultiplex control, which is capable of controlling an apparatus not only synchronously with reproduction of video voice but also synchronously with the video voice to be reproduced.
The above-described objective of the present invention is accomplished by a multiplex and demultiplex controlling apparatus having:
multiplex and demultiplex means for demultiplexing multiplexed image data and control data from each other, adding time information to the above-described demultiplexed image data and control data, respectively, and outputting them;
storage means for temporarily storing an output from the above-described multiplex and demultiplex means;
timing controlling means for generating read-out indication of corresponding image data and control data in accordance with synchronous timing corresponding to the above-described time information which is added to the above-described image data and control data stored in the above-described storage means;
image reproducing means for reproducing an image data read out from the above-described storage means in response to the above-described read-out indication of the above-described timing controlling means, and conducting a feedback output of time information to be added to the above-described reproduced image data;
control processing means for conducting a control processing in accordance with a control data which is read out from the above-described storage means in response to the above-described read-out indication of the above-described timing controlling means, and conducting a feedback output of time information to be added to the above-described control data; and
timing correcting means for correcting the above-described synchronous timing of the above-described timing controlling means in response to a difference between time information fed back from the above-described image reproducing means and control processing-means, respectively.
In addition, as a preferable embodiment, the above-described image reproducing means is characterized in that it includes means for reproducing an image data consisted of a video data and a voice data, and synchronously reproducing the above-described video data and voice data, and also, the above-described multiplex and demultiplex means is characterized in that it discriminates between an image data and a control data from a packet to be transmitted and demultiplex an image data stream and a control data stream from each other, and adds time information to the above-described demultiplexed image data stream and control data stream, respectively.
Furthermore, in the above-described multiplex and demultiplex controlling apparatus, plural kinds of image data or control data to be multiplexed are provided, and the above-described plural kinds demultiplex multiplexed image data or control data from each other, and a control processing synchronous with plural kinds of image reproductions or a plurality of control processings synchronous with a single image reproduction can be conducted in accordance with the above-described demultiplexed plural kinds of image data or control data.
The above-described objective of the present invention is accomplished by a multiplex and demultiplex controlling system having:
means for adding time information to an image data;
means for adding time information to a control data;
multiplexing means for multiplexing the above-described image data to which the above-described time information is added and the above-described control data to which the above-described time information is added;
demultiplex means for demultiplexing the above-described multiplexed image:data and control data from each other;
storage means for temporarily storing an output from the above-described demultiplex means;
timing controlling means for generating a timing signal for indicating read-out of the above-described stored image data and control data in accordance with synchronous timing corresponding to the above-described time information which is added to the above-described image data and control data stored in the above-described storage means;
image reproducing means for reproducing an image data read out from the above-described storage means in response to the above-described timing signal, and conducting a feedback output of time information which is added to the above-described reproduced image data;
control processing means for conducting a control processing in accordance with a control data which is read out from the above-described storage means in response to the above-described timing signal, and conducting a feedback output of time information which is added to the above-described control data; and
timing correcting means for correcting the above-described synchronous timing of the above-described timing controlling means in response to a difference between time information fed back from the above-described image reproducing means and the above-described control processing means, respectively.
Also, the above-described objective of the present invention is accomplished by a multiplex and demultiplex controlling method, having steps of:
demultiplexing multiplexed image data and control data from each other, and adding time information to the above-described demultiplexed image data and control data, respectively;
storing the above-described demultiplexed image data and control data to which time information is added;
generating read-out indication of the above-described stored image data and control data in accordance with synchronous timing corresponding to the above-described time information which is added to the above-described stored image data and control data;
reading out the above-described stored image data in response to the above-described read-out indication;
reproducing the above-described image data which is read out, and conducting feedback of time information which is added to the above-described image data;
reading out the above-described stored control data in response to the above-described read-out indication;
conducting a control processing in accordance with the above-described control data which is read out, and conducting feedback of time information which is added to the above-described control data; and
correcting timing of the above-described read-out indication in response to a difference between time information of the above-described image data which is fed back and time information of the above-described control data which is fed back.
Also, the above-described objective of the present invention is accomplished by a multiplex and demultiplex controlling method, having steps of:
adding time information to an image data;
adding time information to a control data;
multiplexing the above-described image data to which the above-described time information is added and the above-described control data to which the above-described time information is added;
transmitting the above-described multiplexed data;
receiving the above-described multiplexed image data and control data, and demultiplexing the above-described multiplexed image data and control data from each other;
storing the above-described demultiplexed image data and control data;
generating read-out indication of the above-described stored image data and control data in accordance with synchronous timing corresponding to the above-described time information which is added to the above-described stored image data and control data;
reading out the above-described stored image data in response to the above-described read-out indication;
reproducing the above-described image data which is read out, and conducting feedback of time information which is added to the above-described image data;
reading out the-above-described stored control data in response to the above-described read-out indication;
conducting a control processing in accordance with the above-described control data which is read out, and conducting feedback of time information which is added to the above-described control data; and
correcting timing of the above-described read-out indication in response to a difference between time information of the above-described, image data which is fed back and time information of the above-described control data which is fed back.
In the present invention, since multiplexed image data and control data are demultiplexed from each other, and time information is added to the demultiplexed image data and-control data, respectively, and the image data and control data are stored in temporary storage means, and the corresponding image data and control data are read out from the temporary storage means in accordance with synchronous timing corresponding to the time information which is added to the image data and control data, and a control processing is conducted in accordance with the control data while the image data is reproduced, and at the same time, the synchronous timing is corrected in response to a difference between the time information which is added to the image data and the time information which Is added to the control data, it becomes to be possible to control an apparatus not only synchronously with reproduction of video voice but also synchronously with the video voice to be reproduced.